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DETAILED ACTION 

Response to Amendment 

1. This Office Action is responsive to applicant's remarks. Applicant hereby adds new 
claims 37-41 and cancel claim 4. The new claims and amendments are supported at least by 
the teachings of Fig 2, 3 and paragraphs [0031-0037]. 

Continued Examination Under 37 CFR 1.114 

2. The request for a continued prosecution application (CPA) under 37 CFR 1 .53(d) filed on 
[1] is acknowledged. 37 CFR 1.53(d)(1) was amended to provide that the CPA must be for a 
design patent and the prior application of the CPA must be a design application that is complete 
as defined by 37 CFR 1.51(b). See Elimination of Continued Prosecution Application Practice 
as to Utility and Plant Patent Applications, final rule, 68 Fed. Reg. 32376 (May 30, 2003), 
1271 Off. Gaz. Pat. Office 143 (June 24, 2003). Since a CPA of this application is not permitted 
under 37 CFR 1.53(d)(1), the improper request for a CPA is being treated as a request for 
continued examination of this application under 37 CFR 1.114. 



Claim Rejections - 35 USC §103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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4, Claims 1, 2, 33, 35 and 40 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Herman (US 6,075,905) in combination with Peterson US (6,41 1,742 B1). 

Regarding Claim 1: (Currently Amended) Herman teaches a method for blending images into 
a single image (Refer to abstract) comprising: 

selecting two images having overlapping content {Refer to Figure 9, numeral 718); 

dividing the two images into strips a l ong a common p i ano whoro i n oach str i p i s a l ong and 
narrow p ie G e of th e i mago (Refer to column 11, line 12+); 

selecting a strip of un i form w i dth in each of the two images where the two images overlap each 
other ("An effective image processing means for automatically selecting region of each source 
image to be included in the mosaic from the overlapped regions" at column 2,line 58); 
determining differences between the overlapping two strips (Refer to column 9, line 43); 

Herman does not expressly teach determining a line through the overlapping strips where the 
differences between the overlapping strips are minimized. 

Peterson teaches determining a line through the overlapping strips where the differences 
between the overlapping strips are minimized (Refer to column 5,line 17+; also refer to Figure 
3c, 3c- 1 by way of numeral 54); 
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and blending the two images together along the minimized line to create a single image (Refer 
to column 1, line 51). 

Herman and Peterson are combinable because they are in the same field of merging images 
into panoramic images (See title and classification of each invention). 

At the time that the invention was made, it would have been obvious to one of ordinary skill in 
the art to determine a line through overlapping strips where the differences between the 
overlapping strips are minimized and blending the two images together along the minimized line 
to create a single image. 

The suggestion/motivation for doing so would be to "determine the position of one image to 
another image that is arbitrarily positioned relative to that said other image. It also allow the 
operand to blend image that may have arbitrary shapes and sizes." at column 1, line 50+, 
Peterson. 

Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings 
of Herman in combination with Peterson to obtain the specified claimed elements of Claim 1. 

Regarding Claim 33: Claim 33 equally discloses the claimed invention of Claim 1. Claim 33 is 
rejected for the same reasons as listed above at Claim 1 . Claim 33 is the computer readable 
medium encoded with software that resembles the claimed method of Claim 1. Peterson 
discloses ("In general, another aspect of the invention relates to an article that includes a 
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computer readable medium, which stores computer-executable instructions for blending images 
of segments of a view according to the method described above." at column 1, line 46). 

Regarding Claim 2: (Original) Peterson teaches the selected images belong to a set of two or 
more images comprising a scene ("Referring again to FIG. 1, the positioning module 50 of the 
image stitching software 14 determines the relative positions of the segments depicted in two of 
the images 18a-18d so that an image of an object depicted in one of the images can be aligned 
with another image of the same object." at column 3, line 59). 

Regarding Claim 35: (Original) Peterson teaches the two overlapping strips are selected 
according to a mean squared difference algorithm such that the sum of the mean squared 
difference values between the two selected strips is minimized ("For example, the top left comer 
of the doorway is horizontally displaced from the bottom left corner of the image by a distance 
xO in the first images 18a, while it is displaced by a distance x1 in the second image 18a. 
Consequently the second image is displaced to the left of the first image by a distance (d-ieft) 
given by the mathematical equation: d-ieft = x0-x1." at column 4, iine 12). 

Regarding Claim 40: (New) Herman teaches the determining differences comprises 
determining differences between the image data content comprising color space content of the 
overlapping two strips ("The physics of image creation provides strong motivation for affine color 
correction. Under fairly general and natural circumstances, an affine transformation in color 
space compensates for (a) color-space differences between the two images due to different 
acquisition systems;..." at column 15,line 23). 
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5. Claims 3, 34, 37-39 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Herman (US 8,075,905) in combination with Peterson US (6,41 1,742 B1) and further in view of 

Kang et al (US 20030235344 A1 ). 

Regarding Claim 3: (Original) 

Herman and Peterson in combination teaches all the claimed elements as listed above. Herman 
and Peterson in combination does not specifically teach the selected images differ from each 
other based on at least recording time, camera location, camera setting, lighting, shadows, 
and/or background. 

However, Kang teaches the selected images differ from each other based on at least recording 
time, camera location, camera setting, lighting, shadows, and/or background (Refer to 
paragraph [0015]). 

Herman, Peterson and Kang are combinable because they are in the same field of image 
transformations specifically combining image portions. (See title and classification of each 
invention). 

At the time that the invention was made, it would have been obvious to one of ordinary skill in 
the art to select images differing in one of recording time, camera location, camera settings, 
lighting shadows, and/or background. 

The suggestion/motivation for doing so would have been to "have known parameters at a known 
point in space had been used to acquire the image..." at paragraph [0014], Kang. 
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Therefore, it would have been obvious to one of ordinary skill in the art to combine the teaching 
of Herman and Peterson with Kang to obtain all the specified claimed elements of Claim 3. 

Regarding Claim 4: (Canceled) 

Regarding Claim 34: (Original) Kang teaches the software wherein the selected images differ 
from each other based on at least recording time, camera location, camera setting, lighting, 
shadows, and/or background (Refer to paragraph [0015]). 

Regarding Claim 37: (New) Kang teaches wherein the selecting comprises selecting the strips 
of the two images which provide reduced error between the selected overlapping two strips 
compared with non-selected strips of the two images (Refer to paragraph [0007]). 

Regarding Claim 38: (New) Kang teaches the determining differences comprises determining 
differences between image data content of the overlapping two strips (Refer to paragraph 
[0057]). 

Regarding Claim 39: (New) Kang teaches the determining differences between image data 
content comprises determining differences between the image data content of one pixel of one 
of the overlapping two strips and one pixel of another of the overlapping two strips and wherein 
the one pixels of the one and the another of the overlapping two strips both correspond to the 
same subject present in the two images (Refer to paragraph [0050]). 



Application/Control Number: 10/814,302 Page 8 

Art Unit: 2624 

6. Claims 5, 8 are rejected under 35 U.S.C. 103(a) as being unpatentable over Herman 
(US 6,075,905) in combination with Peterson US (6,411,742 B1) and further in view of Peieg 
(US 2003-0076406 A1). 

Regarding Claim 5 (Original): Herman and Peterson in combination teaches all the claimed 
elements as listed above. Herman and Peterson in combination does not specifically teach the 
selected images are divided along a common plane. 

However, Peleg teaches the selected images are divided along a common plane (Refer to 
Figure 6, numeral 604; for an additional example, see Figure 7, numeral 704, paragraph [01 14]). 

Herman, Peterson and Peleg are combinable because they are in the same field of image 
transformations specifically combining image portions. (See title and classification of each 
invention). 

At the time that the invention was made, it would have been obvious to one of ordinary skill in 
the art to teach/disclose the selected images are divided along a common plane. 

The suggestion/motivation for doing so would have been "to address three major issues in 
traditional image mosaicing, image alignment, image cut and paste and image blending." (Refer 
to paragraphs [0031-0035], Peleg). 

Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings 
of Herman and Peterson with Peleg to obtain all the specified claimed elements of Claim 5. 
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Regarding Claim 8: (Original) Peieg teaches cutting the selected images along the minimized 
line (Refer to Figure 4, numeral 405 or refer to Figure 8, numeral 601, item S2 ("dotted dashed 
lines") and numeral 603, item S2 ("dotted dashed lines") and joining the cut images together to 
create the single image of the scene (Refer to paragraph [0053-0055]). 

7. Claims 6, 7, 14, 15 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Herman (US 6,075,905) in combination with Peterson US (6,411,742 B1) and further in view of 

Xiong (US 6,549,651 B2), 

Regarding Claim 6: (Original) Herman and Peterson in combination teaches all the claimed 
elements as rejected above. Herman and Peterson in combination does not specifically teach 
the selected images are divided into strips along one of a vertical plane or a horizontal plane. 

However, Xiong teaches the selected images are divided into strips along one of a vertical plane 
or a horizontal plane ("Further, the present invention, unlike some of the prior art, allows for 
multiple views, from multiple planes and rows of images, and allows for the arbitrary orientation 
of photographic images to be constructed into a panorama, without specialized hardware such 
as a tripod orfisheye lens. In addition, the present system and method can be several orders of 
magnitude faster than the prior art." at column 2, line 27). 

Herman, Peterson and Xiong are combinable because they are in the same field of image 
transformations specifically combining image portions. (See title and classification of each 
invention). 



Application/Control Number: 10/814,302 
Art Unit: 2624 



Page 10 



At the time that the invention was made, it would have been obvious to one of ordinary skill in 
the art to teach/disclose the selected images are divided into strips along one of a vertical plane 
or a horizontal plane. 

The suggestion/motivation for doing so would have been "to provide a registration, calibration 
and global optimization for each of the images being processed." (see column 2, line 2, Xiong) 

Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings 
of Herman and Peterson with Xiong to obtain all the specified claimed elements of Claim 5. 

Regarding Claim 7: (Original) Peterson teaches the two overlapping strips are selected 
according to a mean squared difference algorithm such that the sum of the mean squared 
difference values between the two selected strips is minimized (Tor example, the top left corner 
of the doorway is horizontally displaced from the bottom left corner of the image by a distance 
xO in the first images 18a, while it is displaced by a distance x1 in the second image 18a. 
Consequently the second image is displaced to the left of the first image by a distance (d-left) 
given by the mathematical equation: d-ieft = x0-x1." at column 4, line 12). 

Regarding Claim 14 (Original): Peterson teaches the blending of images is performed 
iteratively ("Referring to Figure 2B, image stitching software 14, blends the images 18a-18d so 
to generate a single panoramic image 26..." at column 3, Sine 52), with the blended single image 
being utilized as one of the selected two images to be blended ("If there are more images, the 
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stitching software 14 sets (224) the reference image to be the next image after the current 
image and repeats the process,,," at column 5, line 48). 

Regarding Claim 15: Peterson teaches the method of blending is performed iteratively (Figure 
2B describes the method of blending images iteratively by blending any of images 18a-18d; 
"...image stitching software 14, blends the images 18a-18d so to generate a single panoramic 
image 26..." at column 3, line 52) until all images comprising the scene have been blended into 
a final single image of the scene ("Consequently, additional processing is required to blend the 
images into each other and create the near seamless panoramic image 26 (Figure 2B)." at 
column 4, line 45), 

8. Claims 9-11, 36 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Herman (US 6,075,905) in combination with Peterson US (6,41 1 ,742 B1 ) and further in view of 
Pham et ai. "Color Correction for an Image Sequence", 1995, IEEE, pages 38-42 and Kang et 
al (US 20030235344 A1). 

Regarding Claim 9: (Original) Herman and Peterson in combination teaches all the claimed 
elements as rejected above. Herman and Peterson in combination does not specifically teach 
calculating a squared color difference value for each pixel pair between the overlapping strips; 
converting the squared color difference values into a gray scale image of the overlapping strips, 
wherein the brightest pixels in the gray scale image correspond to the pixels of greatest 
difference between the two overlapping strips sorting the gray scale pixels from largest to 
smallest difference value; for each sorted gray scale pixel, mapping the gray scale pixel to one 
of two regions within the overlapping strip according to the adjacency of the gray scale pixel to 
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the one of the two regions; determining a cut line between the two regions; cutting each 
selected image along the cut line within the overlapping strip of each selected image; and 
combining the two cut selected images along the cut line to form the single image. 

However, Pham teaches calculating a squared color difference value for each pixel pair 
between the overlapping strips (Refer to page 38; "This color correction technique constructs a 
mapping from the registered overlapping regions of two adjacent images in an image sequence. 
It uses multiple regression to minimize color differences. ASso see right column, "Mapping 
Construction", paragraph 2); 

Kang teaches converting the squared color difference values into a gray scale image of the 
overlapping strips, wherein the brightest pixels in the gray scale image correspond to the pixels 
of greatest difference between the two overlapping strips (Refer to paragraph [0012], [0051]); 
sorting the gray scale pixels from largest to smallest difference value; for each sorted gray scale 
pixel, mapping the gray scale pixel to one of two regions within the overlapping strip according 
to the adjacency of the gray scale pixel to the one of the two regions; determining a cut line 
between the two regions (Refer to paragraph [0051]); cutting each selected image along the cut 
line within the overlapping strip of each selected image (Refer to Figure 2, numeral 250); and 
combining the two cut selected images along the cut line to form the single image (Refer to 
Figure 6, numeral 680, paragraph [0098]). 

Herman, Peterson Pham and Kang are combinable because they are in the same field of image 
transformations specifically combining image portions. (See title and classification of each 
invention). 
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At the time that the invention was made, it would have been obvious to one of ordinary skill in 
the art to combine the teachings of Herman, Peterson, Pham and Kang. All of the claimed 
elements were known in the prior art and one skilled in the art could have combined the 
elements as claimed by known methods with no change in their respective functions, and the 
combination of Herman, Peterson, Pham and Kang would have yielded predictable results to 
one of ordinary skill in the art at the time of the invention. 

The suggestion/motivation for doing so would have been "...because the multiperspective plane 
sweep approach described herein is both computationally efficient, and applicable to both the 
case of limited overlap between the images used for creating the image mosaics, and to the 
case of extensive or increased image overlap." at paragraph [0051], Kang. 

Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings 
of Herman and Peterson with Pham and Kang to obtain all the specified claimed elements of 
Claim 9. 

Regarding Claim 36: (Original) Pham teaches calculating a difference value for each pixel pair 
between the two overlapping strips {Refer to page 38; "This color correction technique 
constructs a mapping from the registered overlapping regions of two adjacent images in an 
image sequence, it uses multiple regression to minimize color differences. Also see right 
column, "Mapping Construction", paragraph 2); 
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Kang teaches converting the calculated difference values into a gray scale image of the 
overlapping strips, wherein the brightest pixels in the gray scale image correspond to the pixels 
of greatest difference between the two overlapping strips (Refer to paragraph [0012], [0051]); 
sorting the gray scale pixels from largest to smallest difference value; for each sorted gray scale 
pixel, mapping the gray scale pixel to a first region or a second region within the overlapping 
strip according to the adjacency of the gray scale pixel to the first region or the second region 
(Refer to paragraph [0051]); determining a cut line within the overlapping strips between the first 
mapped region and the second mapped region(Refer to Figure 2, numeral 250); cutting each 
selected image along the cut line of the overlapping strip of each selected image (Refer to 
Figure 2, numeral 250); and combining the two cut selected images along the cut line to form 
the single image {Refer to Figure 6, numeral 660, paragraph [0098]). 

Herman, Peterson Pham and Kang are combinable because they are in the same field of image 
transformations specifically combining image portions. (See title and classification of each 
invention). 

At the time that the invention was made, it would have been obvious to one of ordinary skill in 
the art to combine the teachings of Herman, Peterson, Pham and Kang. All of the claimed 
elements were known in the prior art and one skilled in the art could have combined the 
elements as claimed by known methods with no change in their respective functions, and the 
combination of Herman, Peterson, Pham and Kang would have yielded predictable results to 
one of ordinary skill in the art at the time of the invention. 
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The suggestion/motivation for doing so would have been "...because the multiperspective plane 
sweep approach described herein is both computationally efficient, and applicable to both the 
case of limited overlap between the images used for creating the image mosaics, and to the 
case of extensive or increased image overlap." at paragraph [0051], Kang. 

Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings 
of Herman and Peterson with Pham and Kang to obtain all the specified claimed elements of 
Claim 36. 

Regarding Claim 10: (Original) Peterson teaches the cut line is determined between a first 
region and a second region to which the pixels have been mapped ("The second image is 
divided into the first and second section by a dividing line that is determined based on an outline 
of the first image. . ." at column 2, Sine 22). 

Regarding Claim 11: (Original) Peterson teaches the cut line corresponds to the line of best 
match between the overlapping strips ("For example, a seam 62 is created... where the two 
images 18c and 18d join each other... and create a near seamless panoramic image 26 (Figure 
2b) ", at column 4, line 42). 

9. Claims 12, 13 are rejected under 35 U.S.C. 103(a) as being unpatentable over Herman 
(US 6,075,905) in combination with Peterson US (6,411,742 B1) and Pham et al. "Color 
Correction for an Image Sequence", 1995, IEEE, pages 38-42 further in view of Kang et al (US 
20030235344 A1) and further in view of Xiong (US 5,549,651 B2). 
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Regarding Claim 12: (Original) Herman, Peterson, Pham and Kang in combination 
disclose/teach all the claimed elements as rejected above. Herman, Peterson, Pham and Kang 
in combination does not expressly teach at least one of the cut images is warped along the cut 
line to improve the fit between the two cut images along the cut line. 

Xiong teaches at least one of the cut images is warped along the cut line to improve the fit 
between the two cut images along the cut line ("The present invention is designed to calibrate 
and align all such 2D rectilinear images with respect to one another and globally, blend the 
images where they overlap, and construct a reconstructed and relatively error free 3D 
panorama image, shown conceptually in 2D form as FIG. 1(b), for any arbitrary geometry." at 
column 3, line 47). 

At the time that the invention was made, it would have been obvious to one of ordinary skill in 
the art to calculate at least one of the cut images as to warp along the cut line to improve the fit 
between the two cut images along the cut line. 

Herman, Peterson, Pham, Kang and Xiong are combinable because they are in the same field 
of creating high quality virtual panoramas. (See abstract Xiong). 

The suggestion/motivation for doing so would have been to "yield dramatic improvements during 
the mathematical, authorization processes and projection cycles with speeds up to several order 
of magnitude faster than any prior system." (Abstract, Xiong). 
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Therefore, it wouid have been obvious to one of ordinary skill in the art to combine the claimed 
elements of Herman, Peterson, Pham, Kang and Xiong to obtain the specified claimed elements 
of Claim 12. 

Regarding Claim 13: (Original) Xiong teaches a Gaussian function is used to warp the at least 
one cut image ("The third step re-projects a!! images... by employing Laplacian pyramid based 
blending using a Gaussian blend mask..." at column 2, Sine 18). 

10. Claim 16 is rejected under 35 U.S.C. 103(a) as being unpatentable over Peterson US 
(6,41 1 ,742 B1 ) and Kang et ai (US20030235344 A1 ). 

Regarding Claim 16: (Currently Amended) Peterson teaches a method for blending images 
into a single image (Refer to column 1, iine 6) 

determining a line through the selected overlapping strips where differences between the 
selected overlapping strips are minimized (Figure 3c; "The dividing iine determiner 54 
determines an outline 74 (Fig. 3c)... formed by aligning the current image 18b' and the reference 
image 18a..." at column 5, line 25 (e.g. Figure 3a, numeral 214 and 216)); blending the two 
images along the determined minimized line to create a 
single image (Refer to column 1, iine 51); 

Peterson does not expressly teach dividing two images into strips along a common plane and 
wherein the selecting comprises selecting the overlapping strips which have reduced error 
between the selected overlapping strips compared with non-selected overlapping strips of the 
two images and warping the single image to minimize blurring along the blending line. 
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Kang teaches dividing two images into strips along a common plane whoro i n each str i ps aro 
l ong and narrow p ie c e of tho i mago; (Refer to paragraph [0011]; further at Figures 3a and 3b) 
selecting a strip of un i form w i dth in each image where the two images overlap (Refer to Figure 
6, numeral 600), wherein the selecting comprises selecting the overlapping strips which have 
reduced error between the selected overlapping strips compared with non - selected overlapping 
strips of the two images (Refer to Figure 6)1. 

and warping the single image to minimize blurring along the blending line. ("Even if the camera 
motion is not horizontal, the images are warped or "rectified" to produce an effective horizontal 
camera motion." at paragraph [001 1]). 

Peterson and Kang are combinable because they are in the same field of merging images into 
panoramic images (see title of each invention). 

At the time that the invention was made, it would have been obvious to one of ordinary skill in 
the art to divide two images into strips along a common plane and select the overlapping strips 
which have reduced error between the selected overlapping strips compared with non-selected 
overlapping strips of the two images and warping the single image to minimize blurring along 
the blending line. 

The suggestion/motivation for doing so would be that "better mosaicing will result if the 
boundaries of the strip are taken to be approximately perpendicular to the "optical flow" (local 
image displacement) generated by the camera motion. Examples are camera translations: 
sideways motion, forward motion, and a genera! translation; as well as camera zoom." @ 
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paragraph [01 1 1]. Warping also allows the user of this method to form, shape, bend and stretch 
the images to obtain a best match for providing a seamless match of a blended single image. 

Therefore, it would have been obvious to one of ordinary skil! in the art to combine the teachings 
of Peterson with Kang to obtain the specified claimed elements of Claim 16. 

1 1 . Claims 17-20 are rejected under 35 U.S.C. 103(a) as being unpatentable over Peterson 
US (6,41 1 ,742 B1 ) and Kang et ai (US20030235344 A1 ) and further in view of Xiong (US 
5,549,651 B2). 

Regarding Claim 17 (Currently Amended): Peterson and Kang in combination disclose/teach 
all the claimed elements as rejected above. Peterson and Kang in combination does not 
expressly teach at least one of the cut images is warped along the cut line to improve the fit 
between the two cut images along the cut line. 

Xiong teaches the minimized line is determined by calculating mean squared difference values 
for pairs of pixels between the two selected overlapping image strips. ("Different combinations of 
overlapping areas are tried to achieve the optimal overlap between images (or, equivaiently, the 
smallest error in the error function or pair wise objective function described herein) using the 
steps described herein, which generally minimizes the average squared pixel intensity (e.g., 
brightness and contrast) difference with respect to certain transformation parameters." at 
column 4, line 65). 



Application/Control Number: 10/814,302 Page 20 

Art Unit: 2624 

At the time that the invention was made, it would have been obvious to one of ordinary skill in 
the art to calculate the minimized line is determined by calculating mean squared difference 
values for pairs of pixels between the two selected overlapping image strips. 

Peterson, Kang and Xiong are combinable because they are in the same field of creating high 
quality virtual panoramas. (See abstract Xiong). 

The suggestion/motivation for doing so would have been to "yield dramatic improvements during 
the mathematical, authorization processes and projection cycles with speeds up to several order 
of magnitude faster than any prior system." (Abstract, Xiong). 

Therefore, it would have been obvious to one of ordinary skill in the art to combine the claimed 
elements of Peterson, Kang and Xiong to obtain the specified claimed elements of Claim 17. 

Regarding Claim 18 (Currently Amended): Xiong teaches at least one of the images is warped 
where the differences between the selected overlapping strips along the blending line exceed a 
predetermined threshold (Where the images overlap there is potential for misalignment when 
constructing a 3D panorama, as indicated by blurry lines 112, for a variety of reasons, ... 
distortions that occur when warping a 2D image to construct a 3D image space. The present 
invention is designed to calibrate and align all such 2D rectilinear images with respect to one 
another and globally, blend the images where they overlap, and construct a reconstructed and 
relatively error free 3D panorama image, shown conceptually in 2D form as FIG. 1(b), for any 
arbitrary geometry." at column 3, Sine 41). 
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Regarding Claim 19 (Original): Xiong teaches wherein the single image is warped by 
application of a Gaussian function ("The Gaussian pyramid may be constructed by applying a 
low-pass filter to the blend mask, which dilutes the sharp edges, from linear interpolation 
between the black and white regions of the blend mask, or from other techniques." at column 
15, line 48). 

Regarding Claim 20 (Original): Xiong teaches where the Gaussian 
function is applied iteratively along a plurality of planes and with a plurality of 
magnitudes of warp to determine the best fit between the images. ("The local pair wise 
registration module 222 iterates until the entire Gaussian pyramid is traversed... and working out 
the finest level resolution, as indicated in decision box 324." at column 5, line 27). 

12. Claims 21, 22 are rejected under 35 U.S.C. 103(a) as being unpatentable over Peterson 
US (6,411,742 B1) in combination with Peleg (US 2003-0076406 A1) and Pham et al. "Color 
Correction for an Image Sequence", pages 38-42. 

Regarding Claim 21 (Currently Amended): Peterson teaches computer-based system for 
blending images into a single image (Refer to Figure 1, numeral 10), determine a line through 
the overlapping strips where the sum of the pixel difference values between the overlapping 
strips are minimized (Refer to column 5,line 17+; also refer to Figure 3c, 3c-1 by way of numeral 
54); and blend the two images together along the minimized line to create a single image (Refer 
to column 1, line 51). 
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Peleg teaches: divide two images having overlapping content into strips along a common plane 
wherein each strip is a long and narrow piece of the image having one dimension which is 
greater than another dimension of the respective strip (Refer to Figure 6, numerals 601-604); 
select a strip of uniform width in each of the two images where the two 
images overlap each other (Refer to paragraph [0140]); 

Peterson in combination with Peleg does not specifically teach/disclose determining pixel 
difference values between the overlapping two strips. 

Pham teaches determine pixel difference values between the overlapping two strips (Refer to 
page 38, Section "Mapping Construction"; "We wish to find a mapping that will convert the color 
value of each of the pixels in A 1 to the corresponding pixel color value in B'.") 

Peterson, Peleg and Pham are combinabie because they are in the same field of merging 
images into panoramic images (see title of each invention). 

At the time that the invention was made, it would have been obvious to one of ordinary skill in 
the art to determine pixel difference values between the overlapping two strips. 

The suggestion/motivation for doing so would be that to create an environment that eases 
"multiple regression steps to find the relationship between the registered overlapping regions of 
two adjacent images in a sequence." see page 38, right column, paragraph 2, Pham. 
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Therefore, it wouid have been obvious to one of ordinary skiii in the art to combine the teachings 
of Peterson, Peleg and Pham to obtain the specified claimed elements of Claim 21. 

Regarding Claim 22: (Original) Peterson teaches wherein the two overlapping strips are 
selected according to a mean squared difference algorithm such that the sum of the mean 
squared difference values between the two strips is minimized ("For example, the top left comer 
of the doorway is horizontally displaced from the bottom left corner of the image by a distance 
xO in the first images 18a, while it is displaced by a distance x1 in the second image 18a. 
Consequently the second image is displaced to the left of the first image by a distance (d-ieft) 
given by the mathematical equation: d-left = x0-x1." at column 4, line 12). 

13. Claim 23 is rejected under 35 U.S.C. 103(a) as being unpatentable over Peterson US 
(6,411,742 B1) in combination with Peleg (US 2003-0076406 A1) and Pham et al. "Color 
Correction for an Image Sequence", 1995, IEEE, pages 38-42, and further in view of Kang et al 
(US 20030235344 A1 ). 

Regarding Claim 23: (Original) Peterson in combination with Peleg and Pham teaches all the 
claimed elements as rejected above. Peterson in combination with Peleg and Pham does not 
specifically teach calculating a squared color difference value for each pixel pair between the 
overlapping strips; converting the squared color difference values into a gray scale image of the 
overlapping strips, wherein the brightest pixels in the gray scale image correspond to the pixels 
of greatest difference between the two overlapping strips sorting the gray scale pixels from 
largest to smallest difference value; for each sorted gray scale pixel, mapping the gray scale 
pixel to one of two regions within the overlapping strip according to the adjacency of the gray 
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scale pixel to the one of the two regions; determining a cut line between the two regions; cutting 
each selected image along the cut line within the overlapping strip of each selected image; and 
combining the two cut selected images along the cut line to form the single image. 

However, Pham teaches calculating a squared color difference value for each pixel pair 
between the overlapping strips (Refer to page 38; "This color correction technique constructs a 
mapping from the registered overlapping regions of two adjacent images in an image sequence. 
It uses multiple regression to minimize color differences. ASso see right column, "Mapping 
Construction", paragraph 2); 
and 

Kang teaches convert the squared color difference values into a gray scale image of the 
overlapping strips, wherein the brightest pixels in the gray scale image 
correspond to the pixels of greatest difference between the two overlapping 
strips (Refer to paragraph [0012],[0051], a!so refer to Figure 1); sort the gray scale pixels from 
largest to smallest difference value (Refer to Figure 1, numeral 145, 146); for each sorted gray 
scale pixel, map the gray scale pixel to one of two regions within the overlapping strip according 
to the adjacency of the sort gray scale pixel to the one of the two regions (Refer to paragraph 
[0051], also refer to Figure 1, numeral 15, 146, 135, 136); determine a cut line between the two 
regions (Refer to Figure 2, numeral 250, also refer to Figure 1, numeral 145, 146); cut each 
image along the cut line of the overlapping strip of each image (Refer to Figure 2, numeral 250, 
also refer to Figure 1, numeral 145, 146); and combine the two cut images along the cut line to 
form the single image (Refer to Figure 6, numeral 660, paragraph [0098]). 
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Peterson, Peleg, Pham and Kang are combinable because they are in the same field of image 
transformations specifically combining image portions. (See title and classification of each 
invention). 

At the time that the invention was made, it would have been obvious to one of ordinary skill in 
the art to combine the teachings of Peterson, Peleg, Pham and Kang. All of the claimed 
elements were known in the prior art and one skilled in the art could have combined the 
elements as claimed by known methods with no change in their respective functions, and the 
combination of Peterson, Peleg, Pham and Kang would have yielded predictable results to one 
of ordinary skill in the art at the time of the invention. 

The suggestion/motivation for doing so would have been "...because the multiperspective plane 
sweep approach described herein is both computationally efficient, and applicable to both the 
case of limited overlap between the images used for creating the image mosaics, and to the 
case of extensive or increased image overlap." at paragraph [0051], Kang. 

Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings 
of Peterson with Pham and Kang to obtain all the specified claimed elements of Claim 23. 

14. Claims 24 and 25 are rejected under 35 U.S.C. 103(a) as being unpatentable over 

Peterson US (6,41 1,742 B1) in combination with Peieg (US 2003-0076406 A1) and Pham et 
al. "Color Correction for an Image Sequence", 1995, IEEE, pages 38-42, and further in view of 

Kang et al (US 20030235344 A1) and Xiong (US 5,549,651 B2). 



Regarding Claim 24: (Currently Amended): Peterson in combination with Peleg, Pham and 
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Kang in combination disclose/teach all the claimed elements as rejected above. Peterson in 
combination with Peleg, Pham and Kang does not expressly teach the cut line is determined by 
calculating mean squared difference values for pairs of pixels between the two selected image 
strips. 

Xiong teaches the cut line is determined by calculating mean squared difference values for pairs 
of pixels between the two selected image strips ("Different combinations of overlapping areas 
are tried to achieve the optimal overlap between images (or, equivalently, the smallest error in 
the error function or pair wise objective function described herein) using the steps described 
herein, which generally minimizes the average squared pixel intensity (e.g., brightness and 
contrast) difference with respect to certain transformation parameters." at column 4, line 65). 
At the time that the invention was made, it would have been obvious to one of ordinary skill in 
the art to calculate the minimized line is determined by calculating mean squared difference 
values for pairs of pixels between the two selected overlapping image strips. 

Peterson, Peleg, Pham, Kang and Xiong are combinable because they are in the same field of 
creating high quality virtual panoramas. (See abstract Xiong). 

The suggestion/motivation for doing so would have been to "yield dramatic improvements during 
the mathematical, authorization processes and projection cycles with speeds up to several order 
of magnitude faster than any prior system." (Abstract, Xiong). 
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Therefore, it wouid have been obvious to one of ordinary skill in the art to combine the claimed 
elements of Peterson, Peleg, Pham, Kang and Xiong to obtain the specified claimed elements 
of Claim 24. 

Regarding Claim 25 (Original) Xiong teaches wherein at least one 
of the images is warped where the differences between the selected strips along 
the cut line exceed a predetermined threshold (Where the images overlap there is potential for 
misalignment when constructing a 3D panorama, as indicated by blurry lines 112, for a variety 
of reasons, ... distortions that occur when warping a 2D image to construct a 3D image space. 
The present invention is designed to calibrate and align all such 2D rectilinear images with 
respect to one another and globally, blend the images where they overlap, and construct a 
reconstructed and relatively error free 3D panorama image, shown conceptually in 2D form as 
FIG. 1(b), for any arbitrary geometry." at column 3, line 41). 

15. Claim 26 is rejected under 35 U.S.C. 103(a) as being unpatentable over Peterson US 
(6,411,742 B1) in combination with Peleg (US 2003-0076406 A1) and Pham et al. "Color 
Correction for an Image Sequence", pages 38-42. 

Regarding Claim 26: (Currently Amended) Peterson teaches system for blending images into a 
single image (Refer to Figure 1, numeral 10), means for determining a cut line through the two 
images where the difference values are minimized (Refer to column 5,line 17+; also refer to 
Figure 3c, 3c-1 by way of numeral 54); and means for blending the two images along the cut 
line to create a blended single image (Refer to column 1, line 51). 
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Peleg teaches: means for dividing two images having overlapping content into strip along a 
common plane in at least one region of overlap wherein each strip is a long and narrow piece of 
the image having one dimension which is greater than another dimension of the respective strip 

(Refer to Figure 6, numerals 601-604); 

Pham teaches means for calculating difference values between the image data content 
of respective pixels of the two images in corresponding strips of uniform length in the at least 
one region of overlap (Refer to page 38, Section "Mapping Construction"; "We wish to find a 
mapping that will convert the color value of each of the pixels in A 1 to the corresponding pixel 
color value in B'.") 

Peterson, Peleg and Pham are combinable because they are in the same field of merging 
images into panoramic images (see title of each invention). 

At the time that the invention was made, it would have been obvious to one of ordinary skill in 
the art to calculate difference values between the image data content 
of respective pixels of the two images in corresponding strips of uniform length in the at least 
one region of overlap. 

The suggestion/motivation for doing so would be that to create an environment that eases 
"multiple regression steps to find the relationship between the registered overlapping regions of 
two adjacent images in a sequence." see page 38, right column, paragraph 2, Pham. 

Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings 
of Peterson, Peleg and Pham to obtain the specified claimed elements of Claim 26. 
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16. Claim 27 is rejected under 35 U.S.C. 103(a) as being unpatentable over Kang et ai (US 
20030235344 A1). 

Regarding Claim 27: (Currently Amended) Kang teaches a system for blending images into a 
single image ("A system and method for deghosting mosaics provides a novel multiperspective 
plane sweep approach for generating an image mosaic from a sequence of still images, video 
images, scanned photographic images, computer generated images, etc." at abstract), 
comprising: a first computing module dividing two images having overlapping content into strips 
along a common plane in at least one region of overlap wh e r ei n e ach str i p i s a l ong and narrow 
p ie GO of tho i mago (Refer to Figure 1, numeral 145, 146, also refer to Figure 2, numeral 200);-a 
second computing module calculating difference values between the 
pixels of the two images i n corr e spond i ng str i ps of un i form w i dth in the at least 
one region of overlap (Refer to paragraph [0058], also Figure 2, numeral 230), wherein the 
difference values individually correspond to a difference of image data content between a pair of 
corresponding pixels of the two images (Refer to paragraph [0057]);_a third computing module 
determining a cut line through the two images where the difference values are minimized (Refer 
to Figure 1, numeral 145, 146, also Figure 2, numeral 230); and a fourth computing module 
blending the two images along the cut line to create a blended single image (Refer to Figure 1, 
numeral 145, 148; paragraph [0080], also Figure 5). 

Kang does not specifically/expressly disclose the claimed elements of the system with the 
equivalent subject matter as claimed in Ciaim 27. However, Kang does teach the computing 
modules that support the system claimed limitations of Claim 27. Ail of the claimed method 
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steps have been rejected by one of the claimed combination as rejected above. Claim 27 stands 
rejected based upon the functionality of the system of Ciaim 27 as taught and fully supported by 
the disclosure of Kang. Claims 1 and 21 equally resemble the claimed subject matter of Ciaim 
27, therefore, claim 27 is also rejected for those reasons as stated above. 

Regarding Claim 41: (New) Peterson teaches the pairs of the pixels individually correspond to 
the same subject present in the two images ("Referring again to FIG. 1, the positioning module 
50 of the image stitching software 14 determines the relative positions of the segments depicted 
in two of the images 18a-18d so that an image of an object depicted in one of the images can 
be aligned with another image of the same object." at column 3, line 59). 

17. Claim 28-32 are rejected under 35 U.S.C. 103(a) as being unpatentable over Kang et al 
(US 20030235344 A1) in combination with Peterson US (6,411,742 B1). 

Regarding Claim 28 (Original): Kang teaches all the claimed element as rejected above. Kang 
does not specifically teach selecting two overlapping strips according to a mean squared 
difference algorithm such that the sum of the mean squared difference values between the two 
strips is minimized. 

Peterson teaches a system (Refer to Figure 1, numeral 10) including selecting two overlapping 
strips according to a mean squared difference algorithm such that the sum of the mean squared 
difference values between the two strips is minimized. (Performed by the portion of Figure 1, 
numeral 50) "If the outlines of the aligned images intersect at more than two points, the dividing- 
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line determiner 54 seiects the two intersection points that are furthest apart from each other to 
define the dividing line 80," at column 5, iine 32). 

Kang and Peterson are combinabie because they are in the same field of image 
transformations, specifically, combining image portions. (See abstract and classification of each 
invention). 

At the time that the invention was made, it would have been obvious to one of ordinary skill in 
the art to select two overlapping strips according to a mean squared difference algorithm such 
that the sum of the mean squared difference values between the two strips is minimized. 

The motivation/suggestion for doing so would have been "to save processing time without 
altering the images and masking out portions of an image." at column 1, line 50+, Peterson. 

Therefore, it would have been obvious to one of ordinary skill in the art to combine the 
teachings/disclosures of Kang in combination with Peterson to obtain the specified claimed 
elements of Claim 28. 

Regarding Claim 29: (Original) Peterson teaches the system (Refer to Figure 1, numeral 10) 
including: a fifth computing module cutting the two images along the cut line (Performed by the 
portion of Figure 1, numeral 54); and a sixth computing module joining the cut images together 
to create the single image (Performed by the portion of Figure 1, numeral 58; "Determining the 
position of the segment depicted in the second image relative to the segment in the first allows 
the method to blend images that may represent segments of the view that are arbitrarily 
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positioned relative to each other. It also allows the method to blend images that may have 
arbitrary shapes and sizes." at column 1, line 51). 

Regarding Claim 30: (Original) Peterson teaches the system (Refer to Figure 1, numeral 10) 
wherein at least one of the cut images is warped along the cut line to improve the fit between 
the two images along the cut line (Performed by Figure 1, numeral 56; "Refer to Figure 2b; 
"Thus, the image stitching software 14 allows a user to biend multiple images 18a-18d to create 
a panoramic image 26 with a field of view that is larger than the field of any one of the multiple 
images." at column 3, line 52). 

Regarding Claim 31: (Original) Peterson teaches the system (Refer to Figure 1, numeral 10) 
the blending of images is performed iteratively, with the blended single image being utilized 

as one of the two images to be blended ('The stitching software 14 checks (222) whether there 
are any more images between the reference image 18a and the current image 18b'. If there are 
more images, the stitching software 14 sets (224) the reference image to be the next image 
after the current reference image and repeats the process of setting a section of the current 
image 18b' invisible 208-220 described above." at column 5, line 46). 

Regarding Claim 32: (Original) Peterson teaches wherein the system is 
included in one of a video camera or a digital camera ("For example, the image 18 to be 
blended may be obtained from a digital camera, storage 16 or a network 28." at column 7, line 
17). 
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Conclusion 

Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Mia M. Thomas whose telephone number is (571)270-1583. The 
examiner can normally be reached on Monday-Thursday 8am-5pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Vikkram Bali can be reached on 571-272-7415. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private 
PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you 
would like assistance from a USPTO Customer Service Representative or access to the 
automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
/Mia M Thomas/ 
Examiner, Art Unit 2624 

/Vikkram Bali/ 

Supervisory Patent Examiner, Art Unit 2624 



